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Abstract. We present an algorithm for model checking temporal-epistemic 
properties of multi-agent systems, expressed in the formalism of inter- 
preted systems. We first introduce a technique for the translation of 
interpreted systems into boolean formulae, and then present a model- 
checking algorithm based on this translation. The algorithm is based on 
obdd’s, as they offer a compact and efficient representation for boolean 
formulae. 


1 Introduction 

Theoretical investigations in the area of multi-agent systems (MAS) have tra- 
ditionally focused on specifications. Various logics have been explored to give 
formal foundations to MAS, particularly for mental attitudes [1] of agents, such 
as knowledge, belief, desire, etc. To consider the temporal evolution of these at- 
titudes, temporal logics such as CTL and LTL [2] have been included in MAS 
formalisms, thereby producing combinations of temporal logic with, for example, 
epistemic, doxastic, and deontic logics. 

Although it is important to investigate formal tools for specifying MAS, the 
problem of verification of MAS must also be taken into account to ensure that 
systems behave as they are supposed to. Model checking is a well-established 
verification technique for distributed systems specified by means of temporal 
logics [3,2]. The problem of model checking is to verify whether a logical for- 
mula i p expressing a certain required property is true in a model M representing 
the system, that is establishing whether or not M j= tp. This approach can also 
be applied to MAS, where in this case M is a semantical model representing 
the evolutions of the MAS, and is a formula expressing temporal- intentional 
properties of the agents. Recent work along these lines includes [4], in which 
Wooldridge et al. present the MABLE language for the specification of MAS. In 
this work, modalities are translated as nested data structures (in the spirit of [5] ) . 
Bordini et al. [6] use a modified version of the AgentSpeak(L) language [7] to 
specify agents and to exploit existing model checkers. For verification purposes, 
both the works of Wooldridge et al. and of Bordini et al. translate the MAS 
specification into a SPIN specification [8] to perform the verification. The works 


of van der Meyden and Shilov [9] , and van der Meyden and Su [10] , are concerned 
with verification of interpreted systems. They consider the verification of a par- 
ticular class of interpreted systems, namely the class of synchronous distributed 
systems with perfect recall. An algorithm for model checking is introduced in the 
first paper using automata, and [10] suggests the use of OBDD’s for this approach. 

The aim of this paper is to present an algorithm for model checking epistemic 
and temporal properties of interpreted systems [11]. This differs from previous 
work by treating all the modalities explicitly in the verification process. We 
focus on temporal-epistemic model checking because the verification of epistemic 
properties (and their temporal evolution) is crucial in many scenarios, including 
communication protocols and security protocols. 

Interpreted systems are a formalism for representing epistemic properties 
of MAS and their evolution with time. The algorithm that we present does 
not involve the translation into existing model checkers, it is fully symbolic, 
and it is based on boolean functions. Boolean functions can be represented and 
manipulated efficiently by means of OBDD’s, as it has been shown for CTL model 
checking [12]. 

The rest of the paper is organised as follows: in Section 2 we briefly review 
OBDD’s-based model checking and the formalism of interpreted systems. In Sec- 
tion 3.1 we present the translation of interpreted systems into boolean formulae, 
while in Section 3.2 we introduce an algorithm based on this translation. We 
provide a proof of the correctness of the algorithm in Section 3.3. We conclude 
in Section 4. 

2 Preliminaries 

2.1 CTL model checking and OBDD’s 

Given a model M and a formula <p in some logic, the problem of model checking 
involves establishing whether or not M j= ip holds. Tools have been built to 
perform this task automatically, where M is a model of some temporal logic [3, 
2,8]. SMV [12] and SPIN [8] are two well-known model checkers; in these tools 
the model is given indirectly by means of a program P. It is not efficient to 
build explicitly the model M represented by P, because M has a size which 
is exponential in the number of variables of P (this fact is known as the state 
explosion problem). Instead, various techniques have been developed to perform 
symbolic model checking, which is the problem of model checking where the model 
M is not described or computed in extension. Techniques for symbolic model 
checking mostly use either automata [8], or OBDD’s [13] for the representation 
of all the parameters needed by the algorithms. For the purpose of this paper, 
we will only consider symbolic model checking of the temporal logic CTL using 
OBDD’s [14], 

CTL is a logic used to reason about the evolution of a system represented as a 
branching path. Given a countable set of propositional variables V = {p, q, ■ ■ •}, 
CTL formulae are defined as follows: 

<p ::= p | -’<£ | p V ip | EXip \ EGip \ E(ipUip) 



where the temporal operator X means in the next state, G means globally and U 
means until. Each temporal operator is pre-fixed by the existential quantifier E. 
Thus, for example, EG(<p) means that “there exists a path in which <p is globally 
true”. Traditionally, other operators axe added to the syntax of CTL, namely 
AX,EF,AF,AG,AU (notice the “universal” quantifier A over paths, dual of 
E). These operators can be derived from the operators introduced here [2], The 
semantics of CTL is given via a model M = ( S , R, V, I) where S — {so, si, . . .} 
is a set of states, R C 5 x S is a binary relation, V : V — * 2 s is an evaluation 
function, and I C 5 is a set of initial states. A path it is a sequence of states 
7r = {so, Si, . . .} such that so € I and Vi, (s$, Si+i) € R- A state Sjina path it is 
denoted with 7Tj. Satisfaction in a state is defined inductively as follows: 
s 1= p iff s € V(p), 

s j= EXip iff there exists a path i r such that 7 r* = s and Tr i+1 [= ip, 
s |= EG(p iff there exists a path n such that TZi — s and 7T i+J - |= tp 
for all j > 0. 

s E(<pUip) iff there exists a path 7r such that tt* = s and a k > 0 such 
that TTi+k \= ip and it i+j \= <p for all 0 < j < k. 
obdd’s (Ordered Binary Decision Diagrams) are an efficient representation 
for the manipulation of boolean functions. As an example, consider the boolean 
function a A (6 V c). The truth table of this function would be 8 lines long. Equiv- 
alently, one can evaluate the truth value of this function by representing the 
function as a directed graph, as exemplified on the left-hand side of Figure 1. As 
it is clear from the picture, under certain assumptions, this graph can be simpli- 
fied into the graph pictured on the right-hand side of Figure 1. This “reduced” 
representation is called the OBDD of the boolean function. 
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Fig. 1. OBDD representation for a A (b V c). 
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Besides offering a compact representation of boolean functions, obdd’s of 
different functions can be composed efficiently: in [13] algorithms are provided 
for the manipulation and composition of obdd’s. 

The idea of CTL model checking using OBDD’s is to represent states of the 
model and relations by means of boolean formulae. A CTL formula is identified 
with a set of states, i.e. the states of the model satisfying the formula. As set 



of states can be represented as a boolean formula, each CTL formula can be 
characterised by a boolean formula. Thus, the problem of model checking for 
CTL is reduced to the construction of boolean formulae. This is achieved by 
composing obdd’s, or by computing fix-points of operators on obdd’s; we refer 
to [2] for the details. By means of this approach large systems have been checked, 
including hardware and software components. 

2.2 Interpreted Systems 

An interpreted system is a semantic structure representing the temporal evolu- 
tion of a system of agents. Each agent i (i = {1 , . . . , n}) is characterised by a set 
of local states Li and by a set of actions Acti that may be performed. Actions 
are performed in compliance with a protocol P t : Li — > 2 Acti ; notice that this 
definition allows for non-determinism. A tuple g ~ (li,...,l n ) G L\ x ...,L n , 
where li € Li for each i, is called a global state and gives a snapshot of the sys- 
tem. Given a set 7 of initial global states , the evolution of the system is described 
by n evolution functions 1 : t{ : L\ x . . . x L n x Act\ x ... x Act n —* Li In this 
formalism the environment in which agents “live” is usually modeled by means 
of a special agent E; we refer to [11] for more details. 

The set 7, t, and the protocols P* generate a set of runs. Formally, a run 7r 
is a sequence of global states 7r = (go, 9i, ■ ■ ■) such that go £ I and, for each pair 
(gj,gj+ 1 ) G 7r, there exists a set of actions a enabled by the protocols such that 
t{gj,a) = gj+i- G C ( L\ x ... x L n ) denotes the set of reachable global states. 

Given a set of agents A = {1, . . . ,n} with corresponding local states, pro- 
tocols, and transition functions, a countable set of propositional variables V = 
{p, q , . . .}, and a valuation function for the atoms V : V — » 2 G , an interpreted 
system is a tuple IS = ( G , 7, 77, ~i, . . . , ~ n , V). In the above G is the finite set 
of reachable global states for the system, 7 C G is the set of initial states, and 
77 is the set of possible runs in the system. The binary relation i G A, is 
defined by g g' iff L(g) = U(g'), i.e. if the local state of agent i is the same 
in g and in g' . Some issues arise with respect to the generation of the reachable 
states in the system given a set of protocols and transition relations; since they 
do not influence this paper we do not report them here. 

Interpreted systems semantics can be used to interpret formulae of a temporal 
language enriched with epistemic operators [11]. Here we assume a temporal tree 
structure to interpret CTLK formulae [15]. The syntax of CTLK is defined in 
terms of a countable set of propositional variables V = {p,q, ■ ■ ■} and using the 
following modalities: 

p ::= p | -><p | <p V ip | EXip \ EGip \ E(<pU<p) | Ki<p 

The modalities AX, EF, AF, AG, AU are derived in the standard way. Further, 
given a set of agents P, two group modalities can be introduced: EpP and 
Cptp denote, respectively, that every agent in the group knows g>, and that is 
common knowledge in the group (see [11] for details). 


1 This definition is equivalent to the definition of a single evolution function t as in [11]. 



Given an interpreted system IS , a global state g, and a formula < p, the se- 
mantics of CTLK is defined as follows: 

IS,g\=p i figeV{p), 

IS, g\=^f lSg ft <p, 

IS,g ^iV(0 2 iff 9 (= or g (= ip 2 , 

IS,g |= EX ip iff there exists a run 7r such that 

7r,- = g for some i, and n t+ i ip, 

IS,g \= EGip iff there exists a run tv such that 

TVi — g for some i, and t V j j= <p for all j > i. 

IS,g |= E(ipU'ip) iff there exists a run tv such that 

7 Vi — g for some i, and a k > 0 such that TVi+k (= ip 
and tv j |= (p for alii < j < i + k, 

IS, g )= Kup iff Vy g' implies g' )= <p 

IS, g f= Er<p iff Vo' G G, g g' implies g' (= g> 

IS, g \= Crp iff Vc?' € G, g g' implies g' \= <p 

In the definition above, t Vj denotes , the global state at place j in run tv. 
Other temporal modalities can be derived, namely AX, EF, AF, AG, AU. We 
write IS |= ip if, for every global state g € G, IS, g\= ip. We refer to [11, 15] for 
more details. 

3 A model checking algorithm for CTLK 

The main idea of tins paper is to use algorithms based on OBDD’s to verify tem- 
poral and epistemic properties of multi-agent systems, in the spirit of traditional 
model checking for temporal logics. To this end, it is necessary to encode all the 
parameters needed by the algorithms by means of boolean functions, and then 
to represent boolean functions by means of OBDD’s. As this last step can be 
performed automatically using software libraries that are widely available, in 
this paper we introduce only the translation of interpreted systems into boolean 
formulae (Section 3.1). In Section 3.2 we present an algorithm based on this 
translation for the verification of CTLK formulae. 

3.1 Translating an interpreted system into boolean formulae 

The local states of an agent can be encoded by means of boolean variables (a 
boolean variable is a variable that can assume just one of the two values 0 or 
1). The number of boolean variables needed for each agent is nv{i) = \l 0 g 2 \Li\~\ . 
Thus, a global state can be identified by means of N = £ nv{i) boolean variables: 

t 

g — (t>i, • - • ,vn)- The evaluation function V associates a set of global states 
to each propositional atom, and so it can be seen as a boolean function. The 
protocols, too, can be expressed as boolean functions (actions being represented 
with boolean variables (aj , . . . , a_\j) similarly to global states) . 

The definition of f» in Section 2.2 can be seen as specifying a list of conditions 
dp,. . . ,d,k under which agent i changes the value of its local state. Each d,j 
relates conditions on global state and actions with the value of “next” local state 



for i. 


ti — d ; l V ... V (‘i.k 

We assume that the last condition Ci.k of t t prescribes that, if none of the con- 
ditions Cij{j < k) is true, then the local state for i does not change. This 
assumption is key to keep compact the description of an interpreted system, as 
in this way only the conditions that are actually causing a change need to be 
listed. 

The algorithm presented in Section 3.2 requires the definition of a boolean 
function R t (g,g') representing a temporal relation between g and g 1 . Rt(g,g') 
can be obtained from the evolution function ti as follows. First, we introduce a 
global evolution function t: 

t= f\ U = /\ (Ci,i V...VCi ifc4 ) 

i€{l» ...,n} i€{l,...,n } 

Notice that t is a boolean function involving two global states and a joint action 
a = (cq, . ,'.,ajif). To abstract from the joint action and obtain a boolean function 
relating two global states only, we can define R t as follows: 

R t (g,g') iff 3a 6 Act : t(g,a,g') is true and each local action a, £ a is enabled by 
the protocol of agent i in the local state h(g)- 

The quantification over actions above can be translated into a propositional 
formula using a disjunction (see [12,3] for a similar approach to boolean quan- 
tification) : 

Rt{9,9')= V IWs.a.ff') AP(g,a)} 

a€Act 

where P(g, a ) is a boolean formula imposing that the joint action a must be con- 
sistent with the agents’ protocols in global state g. R t gives the desired boolean 
relation between global states. 

3.2 The algorithm 

In this section we present the algorithm SATctlk to compute the set of global 
states in which a CTLK formula p holds, denoted with [[<£>]]. The following are 
the parameters needed by the algorithm: 

— the boolean variables (m, . . . ,v n) and (ax, ... , a^r) to encode global states 
and joint actions; 

— the boolean functions P{(v i, . . . ,vn, a i, ■ ■ ■ ,o,m) to encode the protocols of 
the agents; 

— the function V(p) returning the set of global states in which the atomic 
proposition p holds. We assume that the global states are returned encoded 
as a boolean function of (m, . . . , ujv); 

— the set of initial states 7, encoded as a boolean function; 

— the set of reachable states G. This can be computed as the fix-point of the 
operator r = ( I(g ) V 3g' (R t (g' , g) A Q(g')) where 1(g) is true if g is an initial 
state and Q denotes a set of global states. The fix-point of r can be computed 
by iterating r(0) by standard procedure (see [12]); 



— the boolean function R t to encode the temporal transitions; 

— n boolean functions Ri to encode the accessibility relations (these func- 
tions are easily defined using equivalence on local states of G). 

— the boolean function Rg to encode defined by = A -R<- 

*€r 

The algorithm is as follows: 


SATctlk{p) { 

is an atomic formula: return V(p)', 
p is — ><yOi: return G \ SAT C tlk{<Pi )'> 
p is <pi A (f2'. return SATctlk{p i)n 

SATctlk (^2); 

is EXp 1: return EXctlk{pi)\ 
p is E((piUip>2 ): return EU C tlk{p i,P2)\ 
p is EGpr. return EGctlk(pi)\ 
p is Rupp, return ■Kcrz,Jc( ( PiA); 

<p is Epp\: return Ectlk(pi,E)\ 
p is C r p 1: return Cctlk{p\,E)\ 

J 

In the algorithm above, EXqtlk, EGctlk , EUctlk are the standard 
procedures for CTL model checking [ 2 ] in which the temporal relation is R t 
and, instead of temporal states, global states are considered. The procedures 
Kctlk{p, t) and Eqxlk (p, E) and Cctlk{p-> E) are presented below. 

Kctlk{pA ) { 

X = SATctlk(~ v J>)\ 

Y = {g G G|JXi(g,5') and g' € X} 
return ->Y; 

} 


Ectlk(p,E ) { 

X = SATctlk(~ i p)\ 

Y = {g G G\Rf(g,g') and g' e X) 
return -fY; 

} 


CcTLKiViE) { 

X = SATctlk (p)', 

Y =G; 

while ( X i= Y ) { 

X = Y" 

Y — {g e G\Rf(g,g') and g' eY and g' £ SATctlk{p)} 
} return Y; 

} 






The procedure CcTLxiFi -O is based on the equivalence [11] 

Cp<p = Ep(<p A Cp<p) 

which implies that [[CrV 5 ]] is the fix-point of the (monotonic) operator t(Q) = 
[[Ep(<pA (Q))]]. Hence, [[Cpp]) can be obtained by iterating r(G). 

Notice that all the parameters can be encoded as obdd’s. Moreover, all the 
operations inside the algorithms can be performed on obdd’s as presented in [13]. 

To check that a formula holds in a model, it is enough to check whether or 
not the result of SATctlk is equivalent to the set of reachable states. 


3.3 Correctness of the algorithm 

The algorithm presented in Section 3.2 is sound and complete. 

Theorem 1. For every CTLK formula ip, IS \= <p iff SATctlk(<p) = G. (i.e. 
iff the set of states computed by the algorithm is the set of reachable states G). 

Proof. (=>): by induction on the structure of <p. We consider here the epistemic 
operators (a proof for the temporal operators can be found in [2]). Let c p = Kff ip) 
and let IS,g |= Ki(ip). This means that IS,g' f= ip for all g' G G s.t. g g' . 

By the induction step, g' G [[^]]; also we have Ri(g,g') by definition of Ri. This 
implies that g € [[Ki(ip)]\, i.e. g G [[<p]]. The proof for Ep is similar. The proof of 
correctness for common knowledge follows from the correctness of the fix-point 
characterisation of Cr[ll]. 

(<=): straightforward, as the induction steps above are symmetrical. □ 


4 Conclusion 

Temporal logic model checking using OBDD’s [12] is one of the most successful 
techniques for the verification of distributed systems. In the last decade, this 
methodology has been used for the verification of both software and hardware 
components. 

In this paper we have presented an algorithm for the verification of temporal- 
epistemic properties based on the manipulation of boolean functions. The method- 
ology presented here encodes directly a MAS (specified in the formalism of in- 
terpreted systems) by means of boolean formulae; then, the algorithm allows for 
the (fully symbolic) verification of temporal-epistemic properties. Moreover, the 
algorithm allows for the verification of two group modalities (Ep and Cp) and 
is not restricted to a particular class of interpreted systems, nor to a particular 
class of formulae. We are currently implementing the algorithm and in the future 
we aim at testing epistemic and temporal properties of various scenarios from 
the MAS literature. This will help in evaluating the efficiency of the algorithm. 
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